Starter Template for Bootstrap

User Interface

Our project should be displayed on the Interface. The interface of the game includes two layers. The first one is a “welcoming” interface. It contains a “Start” button. When players click the “Start” button, it will enter the second layer. The second screen layer displays the game countdown time, the score of each game, the number of games won by each player, the position of the soccer detected by the camera and so on. There are also two buttons which are the “Restart” button and the “Quit” button. When one round of the game is over, the player can click the “ Restart” button to restart the game, and then the countdown timer will restart, the scores of the game will become zero and the number of games won by the winner of the previous game is increased by one. Also, if the players don’t want to play the game, they can click the “Quit” button to quit the game. Besides, the physical buttons on the PiTFT are also used in this project. “Restart” button is GPIO 23 and the “Quit” button is GPIO 27.

User Interface 1

User Interface 2

The frame of the Interface codes are based on Lab2. We use the pygame library to display the information on the screen. Since there are two screen layers in this project, we need to use a while loop to display them sequentially. The issues we met were that we cannot display several 0s on the screen at the beginning. Since at the beginning of the game, the score should be 0:0. However, there was only one zero displayed on the screen. We tried different numbers and changed the position of the numbers, but it still didn’t work. Then we noticed that the python dictionary cannot have two items with the same key. That was the reason why 0s can not be displayed. Then we exchanged the keys and the values, and exchanged the content of the for loop to display the key. Several zeros were displayed on the screen.

Break Beam Sensors

Our project uses infrared sensors to detect the goals of soccer games. For the infrared sensors, we used the break beam sensors to detect the soccer ball goals. The break beam sensors are divided into two parts. One is the emitter: it emits a beam of infrared light. The other is the receiver: when pointed at the emitter, receives the beam of infrared. We put a pair of infrared sensors on the sides of the goal. If goals are scored, the infrared ray will be broken and the sensors will detect them. Both the emitter and the receiver are powered by a 5V power source, and the following diagram shows the wiring of the sensors and Raspberry.

For the software part, we try to first display the status of the beam. Then we modified the codes to count the goals when the beam was broken. We met an issue in this part. We need to modify the codes to make sensors detect the goals. So we need to set two variables to express the scores of two players. But when we tested our codes, the scores could not change even if the goals were made. We found that we need to set the two variables as global variables so that they can change in the callback function.

Audio

Our project contains the sound part. When the goals are made, the cheer sound will be produced. The sound equipment was provided by lab 239. For the software part, we used the mixer function of the pygame library to play sound when goals were made. The issue we met in this part is that if we use sudo command, we cannot play sound in the Raspberry. We need to use the “python” command instead of the “sudo python” command to play sound. We still cannot figure out the reason so we use the “python” command to run our program.

Camera Module (OpenCV ball detection)

For the camera module, we decided to use OpenCV to realize the function of red ball detection. We first grab the current frame of the camera's video stream. Then, we resize each frame and using OpenCV's algorithm to remove the noises. After that, we used cv2.cvtColor() function to get its HSV color space for better recognizing the color of the ball.

After getting its mask, we used morphology opening operation to the mask to remove small spaces to better recognize the circle of the ball. By using cv2.findContours function, we can get a set of contours of the potential ball locations. Basing on that contours set, we can use the cv2.minEnclosingCircle() function to get the most probable location of the ball.

        vs = VideoStream(src=0).start()
        frame = vs.read()
        frame = imutils.resize(frame, width=600)
        (h, w, c) = frame.shape
        print ('width:  ', w)
        print ('height: ', h)
        print ('channel:', c)
        blurred = cv2.GaussianBlur(frame, (11, 11), 0)
        hsv = cv2.cvtColor(blurred, cv2.COLOR_BGR2HSV)
        mask = cv2.inRange(hsv, redLower, redUpper)
        mask = cv2.erode(mask, None)
        mask = cv2.dilate(mask, None)
        cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL,
                                cv2.CHAIN_APPROX_SIMPLE)
        cnts = imutils.grab_contours(cnts)
        center = None
        c = max(cnts, key=cv2.contourArea)
        (( x,  y), radius) = cv2.minEnclosingCircle(c)
        M = cv2.moments(c)

The result of the ball detection is shown below:

Goalkeeper's Mechanism Design(Servo motor)

Inspired by the idea from past projects, Sandwich Maker, we created a mechanism design to control the movement of the goalkeeper. As the gif shown below, the rotation of the servo motor is converted to the horizontal movement of the goalkeeper. Though it is not a 100% linear mapping from angle to horizontal shift, we still consider it as a linear mapping.

For instance, the servo motor's angle rotates from 30 degree to 90 degree, corresponding to the goalkeepers move to the left to right. Then we assume that the 60 degree representing the goalkeeper moves to the middle position.

Automatic Goalkeepers

Combining the ball detection function and the servo motor mechanism design, we can get the prototype of the automatic goalkeepers. Here is how the automatic goalkeepers work. As the diagram shows below, the using the ball location(x, y) and the goal position(x2, y2), we can easily calculate the goalkeeper location y1, as the value of x1 is fixed. As mentioned above, we considered the rotation of servo motor mapping to the movement of goalkeeper, y1, in this diagram, linear. Therefore, we can know how many degree should the servo motor rotates.

The overall results of the automatic goalkeepers is shown below. As the red balls go to left, the goalkeeper move to left and if the red balls go to right, the goalkeeper move to right.

Results

At last, the combination of both the hardware design, including camera, sensors, motors control and the User Interface Design led to the success of the final electric foosball table. As the video shows, everything performs as expected. A user can first click the start button on screen to start the game. Then, the camera, break beam sensors and servo motor start working. Once the player makes a goal, the speaker will play the cheering sound and the score of the corresponding team will increase by 1 and display on the PiTFT screen. Besides, the possession rate of each team, the remaining time and the ball position on the camera will be displayed as well.

At the same time, the camera continuously keeps track of the location of the ball, returns the exact locations and delivers it to the Raspberry Pi. After taking the ball location, the location of the goal and keeper into account, it will calculate how much the goalkeeper should move, and the automatic goalkeeper will move to the exact position, waiting to block the ball.

One of the problems of the system is that the location of the balls analyzed by OpenCV cannot always recognize the location of the ball. It is easily affected by the reflection of the light and the edge of the foosball table due to similar color. Therefore, I tried to use prepossessing algorithm in OpenCV and use a stricter thresholds to increase its stability. But as the ball is blocked by the player, it is still not able to recognize the ball, which can be improved later.

Conclusion

Overall design

In conclusion, our project achieved all the goals we set at the beginning. First of all, a start menu allow users to enter the game. As the game start, the system can keep track of both the number of goals for two table foosball players, remaining time, game information such as the location of the ball and the procession of each team. All these information will display on the screen of the PiTFT. Also, a cheering sound after each goal will be played by an audio module connected to the Raspberry Pi. Besides, we achieved the automatic goalkeeper function, which keeps track of the soccer ball movement and move with the ball to block out the shot. After each game, the user can choose to rematch, or to quit the game.

All the function works well and achieves the objectives we set. The only problem occurred to us is that when we tested the audio part, we found that if we used the “sudo python xx.py” command, the sound cannot be played as normal. We need to use the “python xx.py” command instead to run the program.

Overall, the final system is a very successful prototype of the electric foosball table that turns our rough idea into reality and we were able to design and build the hardware, including the most difficult mechanic modules to control the shift of the goalkeeper.

Future work

Firstly, in the UI part, we can add a “Pause” button to stop the countdown timer. If players want to stop the game, they can click the “Pause” button to stop the game.

Secondly, the ball detection was only 95% accurate and easily affected by the reflection of lights. We can use better preprocessing algorithms to solve the problem and use a more stable light source if possible.

Thirdly, the bracket we used to place the camera was not good enough. Therefore, we need to improve the stability of the bracket in the future, which makes sure that the camera is in the exact angle and heights all the time and avoids the adjustment of the camera before the game starts.

Lastly, we need to improve the accuracy of the angle of the servo motor rotating and improve the sensitivity of the movement of the soccer ball. We can use a more accurate motor like step motor to achieve a higher accuracy.

Work Distribution

Yunquan Zhang

yz2793@cornell.edu

Designed the software architecture (including the beam sensors control, audio part, servo motor control, OpenCV ball detection and automatic goalkeeper module) and hardware structure (Mechanism design of the goalkeeper, circuits design of the system).

Yifan Feng

yf372@cornell.edu

The interface part, the sensor part, the countdown part, the machine part of controlling the goalkeeper’s movement, the bracket of the whole project.

Parts List

Raspberry Pi - Provided in Lab
Raspberry Pi Camera V2 - Provided in Lab
Servo Motor - Provided in Lab
IR Break Beam Sensor - 2 pairs - Provided in Lab

Foosball Table/Soccer Game - $29.95

Wires - Provided in lab
Speakers - Provided in lab

Total: $29.95

References

[1]Lab2_Fall2022_v4 on Canvas
[2] Instructions for break beam sensor
[3]Standard-Servo-Product-Documentation-v2.2.pdf
[4] OpenCV and Camera
[5]Python 3
[6]OpenCV 4.6.0
[7]NumPy
[8]Pygame

ECE 5725 Project:Electric Foosball Table

Demonstration Video

Introduction

Project Objective

Basic Objective:

Advanced Objective:

Overall Objective:

Design and Testing